In this post we describe how derivatives are defined in higher dimensions, when dealing with multi-input functions. We explore these ideas first with $N=2$ inputs for visualization purposes, generalizing afterwards.
Press the botton 'Toggle code' below to toggle code on and off for entire this post.
from IPython.display import display
from IPython.display import HTML
import IPython.core.display as di # Example: di.display_html('<h3>%s:</h3>' % str, raw=True)
# This line will hide code by default when the notebook is exported as HTML
di.display_html('<script>jQuery(function() {if (jQuery("body.notebook_app").length == 0) { jQuery(".input_area").toggle(); jQuery(".prompt").toggle();}});</script>', raw=True)
# This line will add a button to toggle visibility of code blocks, for use with the HTML export version
di.display_html('''<button onclick="jQuery('.input_area').toggle(); jQuery('.prompt').toggle();">Toggle code</button>''', raw=True)
In this Section we describe how the notion of a derivative for single-input functions is naturally generalized to multi-input functions.
Instead of the derivative representing the slope of a tangent line in the case of a single-input function, the derivative of a multi-input function represents the set of slopes that define a tangent hyperplane.
This is illustrated in the next Python cell using the following two closely related functions
\begin{array} \ g(w) = 2 + \text{sin}(w)\\ g(w_1,w_2) = 2 + \text{sin}(w_1 + w_2) \end{array}In particular we draw each function over a small portion of its input around the origin, with the single-input function on the left and multi-input function on the right. We also draw the tangent line / hyperplane - generated by the derivative there - on top of each function at the origin.
# plot a single input quadratic in both two and three dimensions
func1 = lambda w: 2 + np.sin(w)
func2 = lambda w: 2 + np.sin(w[0] + w[1])
# use custom plotter to show both functions
callib.derivative_ascent_visualizer.compare_2d3d(func1 = func1,func2 = func2)
Here we can see that the derivative for the multi-input function on the right naturally describes not just a line, but a tangent hyperplane. This is true in general. How do we define the derivative of a multi-input function / the tangent hyperplane it generates?
In post 2 of this series we saw how the derivative of a single input function $g(w)$ at a point $w^0$ was approximately the slope
\begin{equation} \frac{\mathrm{d}}{\mathrm{d}w}g(w^0) \approx \frac{g(w^0 + \epsilon) - g(w^0)}{\epsilon} \end{equation}of the secant line passing through the point $(w^0,\,\,g(w^0))$ and a neighboring point $(w^0 + \epsilon, \,\, g(w^0 + \epsilon))$, and letting $|\epsilon|$ shrink to zero this approximation becomes an equality, and the derivative is precisely the slope of the tangent line at $w^0$.
In the next Python cell we repeat an experiment illustrating this point from an earlier post. Here we plot the function $g(w) = \text{sin}(w)$ over a short window of its input. We then fix $w^0 = 0$, take a point nearby that can be controlled via the slider mechanism, and connect the two via a secant line. When the neighborhood point is close enough to $0$ the secant line becomes tangent, and turns from red to green.
# what function should we play with? Defined in the next line, along with our fixed point where we show tangency.
g = lambda w: np.sin(w)
# create an instance of the visualizer with this function
st = callib.secant_to_tangent.visualizer(g = g)
# run the visualizer for our chosen input function and initial point
st.draw_it(w_init = 0, num_frames = 200)